Fix #5301: Prevent early termination in list-files when directory has many files #5311
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
… many files - Remove early process termination in execRipgrep that was causing incomplete directory traversal - Implement balanced sampling algorithm to ensure fair representation across all directories - Increase timeout values to allow complete directory scanning before applying limits - Add applyBalancedSampling function to distribute file selection evenly across directories - Maintain existing 200-file limit while ensuring all directories are represented This fixes the issue where having 200+ files in one directory (e.g., 'a/') would cause other directories (e.g., 'b/') to be completely ignored in the file listing results.
dosubot
bot
added
size:L
|
✅ No security or compliance issues detected. Reviewed everything up to aa77ada. Security Overview
Detected Code ChangesThe diff is too large to display a summary of code changes. Reply to this PR with |
hannesrudolph
added
the
Issue/PR - Triage
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This PR fixes issue #5301 where the tool would fail to retrieve the complete project structure when one directory contains many files (200+), causing other directories to be completely ignored.
Problem
The issue was in the function in . When the first directory alphabetically (e.g., 'a/') contained 200 or more files, the ripgrep process was being killed early, preventing it from scanning subsequent directories (e.g., 'b/').
Solution
Key Changes
Testing
Before/After
Before: Directory 'a/' with 200 files would cause directory 'b/' to be completely missing from results
After: Both directories 'a/' and 'b/' appear in results with balanced representation
Fixes #5301
Important
Fixes issue #5301 by removing early termination logic, implementing balanced sampling, and increasing timeouts in
list-files.tsto ensure complete directory traversal.listFiles()inlist-files.tsto prevent process from being killed when reaching 200-file limit.applyBalancedSampling()to ensure fair file representation across directories.execRipgrep()from 10,000ms to 15,000ms to allow complete directory traversal.test-reproduce-issue.jsto create a test structure with 200 files in one directory and 3 in another.test-small-case.jsfor a smaller test case with 10 files in one directory and 3 in another.This description was created by
for aa77ada. You can customize this summary. It will automatically update as commits are pushed.